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ABSTRACT 


Recent work at JPL in real-time distributed computation and control has culmi- 
nated in a prototype force-reflecting telemanipulation system having dissimilar master 
(cable-driven force-reflecting hand controller) and slave (PUMA 560 robot with custom 
controller), extremely high sampling rate (1000 Hz), and low loop computation delay (5 
msec). In a series of experiments with this system and five trained test operators cover- 
ing over 100 hours of teleoperation, we measured performance in a series of generic and 
application-driven tasks with and without force feedback, and with control shared between 
teleoperation and local sensor referenced control. Measurements defining task performance 
include 100-Hz recording of six-axis force/torque information from the slave manipulator 
wrist, task completion time, and visual observation of predefined task errors. 

The tasks consisted of high precision peg-in-hole insertion, electrical connectors, velcro 
at tach-de- attach, and a twist-lock multi-pin connector. Each task was repeated three times 
under several operating conditions: “normal” bilateral telemanipulation, forward position 
control without force feedback, and shared control. In shared control, orientation was lo- 
cally servo controlled to comply with applied torques, while translation was under operator 
control. All performance measures improved as capability was added along a spectrum of 
capabilities ranging from pure position control through force-reflecting teleoperation and 
shared control. Performance was maximal for the bare-handed operator. 
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SUMMARY 


An extensive experimental evaluation has been conducted of several modes of teleoperation 
using an advanced telemanipulation system, a representative sampling of tasks, and five test 
operators. The results quantified substantial improvements in three measures of task 
performance as more sophisticated control capabilities were added to the system. 

The JPL Enhanced 6-Axis breadboard (ESAB) system (Bejczy & Szakaly, 1988) is a new 
telemanipulation architecture featuring the JPL force reflecting hand controller as the input device 
(master) and a Puma 560 industrial manipulator fitted with the JPL force/torque sensing smart 
hand as the remote manipulator (slave). Control and coordinate transformation computation is 
performed by distributing the load among two JPL Universal Motor Control systems, one each on 
the master and slave sides. 

The study developed three measures of performance and applied them to data from multiple 
repetitions of four classes of tasks by five test operators. The total evaluation represented over 
100 hours of experimental teleoperation. 

The experiments tested performance of the tasks with 5 modes of teleoperation: 

• position control 

• position control with visual display of force information 

• kinesthetic force feedback 

• shared control 

• direct manual control 

In position control, the slave robot position followed the master and no force/torque 
information was provided to the operator. In visual display of force, force and torque information 
was provided to the operator through a computer graphics display. In kinesthetic force feedback, 
force and torque information was brought back to the operator by applying forces and torques 
directly to his/her hand through an active joystick. In shared control, force information is 
simultaneously fed back to the operator, and used by the machine. Machine intelligence and 
human intelligence cooperate to perform the task. 

Direct manual control was achieved by having the operators perform the task directly without 
the remote manipulation system involved. This experimental condition provided an absolute 
benchmark independent of any particular telemanipulation technology. The other modes were 
provided by the control software of the ESAB. 

The tasks consisted of a mix of basic tasks designed to measure technical capabilities as 
well as to illustrate suitability for applications. The four tasks were: 
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Attach and detach Velcro fasteners 


Velcro is commonly used in space vehicles for keeping parts fixed in the absence of gravity. 
Its hook and loop fastening system has nonlinear mechanical properties which challenge 
manipulation capabilities. 

Insert a peg into a machined hole 

Peg-in-hole is the classic task in robotic interaction with a mechanical environment. It is a 
test of the robot’s ability to achieve ultra-precision positioning in spite of nonlinear mechanics 
and imprecise knowledge. The holes and peg in this experiment were machined to tolerances 
common in automotive bearing practice. 

Manipulate standard electrical connectors 

This task is similar to the peg-in-hole task except that the components were standard off- 
the-shelf electrical signal and power connectors: 120V chassis power cord, 1/4" headphone jack, 
25-pin computer connector. 

Connect a bayonet style multi-pin connector 

This connector was more difficult than the others because it required a rolling motion to 
make/break the physical and electrical connection. 

Performance was evaluated with three measures: completion time, error rate, and sum-of- 
squared-force (SOSF). All of these measures decrease with "better" performance of a given 
task. The first is a purely economic measure applicable to predicting the cost of completing a 
given operation. The last two are quality measures applicable to assessment of the degree of 
risk with which a task can be performed. 

RESULTS 

Some condensed results of the study are presented in Figures i and ii. Task completion time 
and the SOSF force performance measure were computed for the peg-in-hole and the electrical 
connectors tasks. In these computations, data from nine different hole geometries (in the case of 
the peg-in-hole task) and separately, the three electrical connector types (in the case of the 
electrical connector task) were averaged over the five test operators to come up with overall 
performance figures for the five control modes described above. 

For the peg-in-hole task, both completion time and force dropped dramatically as the control 
capability was increased from pure position control to the exquisitely sensitive control of the 
bare human hand. For the electrical connectors, completion time was substantially reduced only 
with shared control, but the force generated to complete the tasks dropped significantly even 
with only visual display of the force information. 
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TIME FORCE 



Figure i 

Completion time and force performance for the peg-in-hole task. The data are averaged over 27 
repetitions by five test operators in each mode. 


TIME FORCE 



Figure ii 

Completion time and force performance for the electrical connectors task. The data are averaged 
over 3 repetitions of insertion and extraction of three standard electrical connectors by five test 
operators. 
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CONCLUSIONS 


This study has quantified the distance yet to be covered before teleoperators will have the 
same dexterity as the bare human hand. We have measured task performance in three 
dimensions: completion time, force performance, and error rate. All of these measures improved 
as capabilities were added to the manipulation system although the improvement varied as a 
function of task. The telerobotic system we tested has an extraordinary number of control 
configurations (at least one million!). Only two of these forms of shared control were tested and 
the resulting improved task performance has shown significant potential for the technology. The 
performance of the unaided human was measured to serve as an upper bound on manipulation 
performance. 

The performance of the unaided operator should be carefully differentiated from the case of an 
EVA astronaut. EVA task performance is reduced due to various factors such as the mechanics 
of the suit and glove, the elimination of tactile sensation, and space physiological effects. For 
mission planning purposes, pre-EVA time spent donning and removing the suit as well as pre- 
breathing, etc. must be amortized over the available work time. The exquisite level of dexterity 
achieved in these experiments by natural human manipulation is a goal to which both telerobot 
and space suit designers can strive. 

Future work will need to address additional constraints and technologies foreseen in space 
applications. Experiments currently under way are addressing the effects of time delays 
interposed between master and slave, and the unique characteristics of dual arm 
telemanipulation. 
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INTRODUCTION 


Teleoperation is expected to perform a wide variety of tasks in future space operations. 
Tasks such as space-station construction will create huge demands for extra-vehicular ac- 
tivity (EVA), which will far exceed available astronaut time. Of course, a major benefit 
of teleoperation is enhanced human safety: the operator is physically removed from the 
actual worksite and the sources of danger, located instead at a control station on earth or 
in a “shirtsleeve” environment on orbit. The price paid for the advantages of teleoperation 
is usually expressed in reduced performance. Thus it is important to devote attention to 
the quantitive evaluation of telerobotic technology in order to assess its benefits in terms 
of mission productivity. 

In order to pursue the goal of a realistic sense of presence at the worksite, many manip- 
ulator designs, starting with Goertz (1954), have incorporated force feedback capabilities. 
For a good review see Goertz (1964). As a result, many of the studies attempting to quan- 
tify the performance of various teleoperator designs have concentrated on the question 
“Can force/torque feedback improve teleoperation performance?” 

Kugatli (1972) studied the effects of a compliant manipulation arm and force feedback 
on manipulation of an inertial load. Tasks performed included the capture of a 400 lb 
pendulum and manipulation of a 650 lb mass through a maze on a flat-floor air-bearing 
table. Results showed force feedback had a large effect on task completion time and error 
rate (i.e., hitting the wall) for the maze task. 

Hill and Salisbury (1977) evaluated three master-slave teleoperators (the Ames Ex- 
oskeletal Master Slave, MA11 & MA23 by Flatau et al. 1972) with an instrumented task 
board which emphasized the peg-in-hole task. Their results documented task completion 
time for the different arms as a function of peg tolerance and showed improved perfor- 
mance when force feedback was present. For a given peg clearance, task completion time 
varied linearly with distance. They also reported bare-handed operator performance. The 
pegs used ranged in size from 1 inch to 1.996 inches and were to be placed in a two-inch 
hole. Hill’s later work (1979) developed a model of teleoperator task completion time from 
industrial time and motion models. 

Other studies include those by Bejczy and Handlykken (1981), who investigated the 
effects of varying force-feedback gain in the novel generalized bilateral teleoperator archi- 
tecture in which the force-reflecting master hand controller (Bejczy & Salisbury 1983) is 
kinematically optimized for man-machine coupling, and real-time computation is provided 
for coordinate transformations between the master and slave joint spaces. 

Recent evaluation studies (Draper et al. 1987) have emphasized on broadening the 
base of measurements against which task performance can be judged. Besides task comple- 
tion time, useful measures included number of task errors, peak force and variance in force 
(see also Hannaford 1987). Although ANOVA did not show a significant completion time 
improvement due to force feedback, the other measures did indicating that force feedback 
allowed the task to be performed with higher-quality results if not at a faster rate. 
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SYSTEM DESCRIPTION 

The JPL teleoperation laboratory has recently developed a unique telemanipulation 
system featuring advanced modes of shared- and supervisory-control-based remote ma- 
nipulation (Szakalay, Kim, & Bejczy 1989). The Enhanced Six-Axis Breadboard (ESAB) 
teleoperation system consists of the JPL-Stanford Force Reflecting Hand Controller (re- 
cently refurbished and upgraded from the original [Bejczy &: Salisbury 1981] design), a 
Puma 560 manipulator, and the JPL Puma Smart Hand (Fiorini 1988) (see Figure la and 
lb). 

The JPL Puma Smart Hand is a single-degree-of-freedom gripping mechanism mount- 
ed in series with a six-axis force-torque sensor. A local microprocessor performs gripping 
control, force-torque sensor processing, and communication functions. These devices are 
integrated and controlled with two JPL universal-control computer systems (Bejczy & 
Szakaly 1987), which perform all coordinate system transformations, master and slave DC 
motor control, and operator interface functions. No part of the Unimation Puma control 
system is used. 

The computation and control system electronics are completely housed in two specially 
modified multibus card cages. The card cages contain NS 32000 family computer boards, 
power supplies, and power amplifiers for the master and slave. Master and slave electronics 
card cages are connected by a parallel cable for bidirectional exchange of position and force 
data. 

The system contains a total of five NS 32016 processors dedicated to forward and 
inverse kinematics computations for the master and slave manipulators. Processors com- 
municate locally via shared memory, and between master and slave sides through the 
parallel cable (Figure 2). 

CAPABILITIES 

The ESAB system can be configured by the user in a wide variety of ways. All control 
modes and gains can be independently selected for each task-space axis. Motion Control 
modes include position control, rate control, and “none.” A “bead diagram” (Figure 3) can 
be used to express the permissible control modes for each axis. In this diagram, a valid 
control mode can be selected by stringing one bead from each column without crossing a 
horizontal bar. The current system thus permits 10 possible control modes per axis. The 
total number of control modes is thus a mind boggling 10 6 modes! In addition, for each 
option column, one or more parameter value must be specified. The ESAB thus provides 
an almost infinite set of control possibilities. 

The P osition- Control mode servo controls the slave position and orientation to match 
the master’s. An indexing function allows slave excursions beyond the 1-cubic-foot work 
volume of the Force- Reflecting Hand Controller (FRHC), and allows the operator to work 
at any task site and orientation from his or her own most comfortable position. 
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Figure 1(a) JPL Enhanced Six- Axis Breadboard System seen from operator control station. 
Operator holds six-degree-of-freedom force-reflecting hand controller through which position 
commands are send to the PUMA 560 industrial manipulator (background). Forces sensed by 
the manipulator hand are transmitted to the operator’s hand via the hand controller. Coordinate 
transformations are performed by a special-purpose computing architecture (not shown). Opera- 
tor views task board with the three task monitors (left). A curtain blocks direct viewing during 
training and experimentation. Also shown are graphics displays: force/torque information 

(bottom right) and real-time solid shaded animation (top right). 
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Figure 1(b) and (c). System components consist of (b) the Puma 560 manipulator, JPL Smart 
Hand, and modular task board; and (c) the force-reflecting hand controller (foreground), control 
electronics (bottom right), and predictive graphics display. The graphics display was not used in 
these experiments. 
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Figure 2. System block diagram of high speed dedicated multi-processor system which per- 
formed coordinate transformations and control of master hand controller and slave robot manipu- 
lator. 
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Master Side 

Slave Side 

Motion Control Mode Force Feedback 

Local Force Accommodation 



Position Scale 
Velocity Scale 
Hand Controller 
Damping 


Force Feedback Scale 
Software-Spring Con- 
stant 


• Virtual Damping Coefficient 

• Virtual Compliance Coefficient 


Figure 3. Control of each axis of the six axis telemanipulation system can be specified by 
conceptually "threading a string of beads" from left to right without crossing any heavy lines. 
Within each column, one or more parameter values may be chosen as appropriate. Key: "Kin. 
F.F.B.," Kinesthetic force feedback, "SWS," Software Spring — a restoring force which returns 
the joystick to the position corresponding to zero velocity. Inset shows typical control modes 
selected for each axis for the experiments in this study. 


6 























Force Feedback can be generated by the hand controller to serve two functions: re- 
flection of task forces and application of a return force to center the hand controller when 
released (this function is referred to as “software spring” or SWS). Force reflection is 
optionally used with position control and SWS with rate control. 

Rate Control sets slave endpoint velocity or angular velocity in task space based on 
the displacement of the FRHC. 

Position and rate modes exist solely on the master side — the slave receives the 
same incremental position commands in either case. In contrast, Force- Accommodation 
mode resides at the slave side. In this mode, in selected degrees of freedom the slave 
computer generates position setpoints based on integration of the sensed force or torque 
in that direction. The effect of this integrated force feedback is to produce a viscous 
mechanical impedance at the end effector. This is equivalent to the generalized damper of 
Whitney (1985). (Recent work has extended this mode to include true compliance, e.g., the 
manipulator emulates a spring and damper in the selected axes.) Force-accommodation 
mode provides a local control capability for complex manipulation tasks such as the peg- 
in-hole task. 

Because of the local force accommodation capability, several forms of “shared control” 
(in which both manual and automatic control coexist) are possible. For example, control 
can be allocated in each task-space degree of freedom in a manner similar to Hybrid Control 
(Raibert Sz Craig 1981) between manual and automatic (local) control. Alternatively, 
control can be shared even in the same axis. In this study, “Shared Force Accommodation” 
shall describe a mode in which the operator’s commands and the local force accommodation 
signal are summed in task-space orientation axes and position control with force feedback 
used in translation axes. “Hybrid Force Accommodation” shall describe a mode in which 
orientation was solely under control of local force accommodation and translation under 
position control by the operator. 

In this study, five control modes plus direct human task performance were experimen- 
tally tested. These are: 

1. Position control without force feedback 

2. Position control with visual force and torque display 

3. Position control with kinesthetic force feedback 

4. Shared Force Accommodation 

5. Hybrid Force Accommodation 

6. Direct human operation 

The same performance measures were computed for all six modes of operation. Direct 
human operation was included to provide a measure of task difficulty which is independent 
of the manipulation system we used. Presumably the tasks used in this study can be related 
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to others by comparing human performance figures. The human performance figures also 
provide a goal point for telemanipulator system performance. Direct human operation 
bears the same relationship to EVA. Direct human operation represents the performance 
point toward which space-glove designers can aim. 

CONTROL STATION 

The master side of the system is installed in a separate control station without a direct 
view of the robot work area. Three television cameras provide top, upper-left rear, and 
right rear views of the task board. The two rear-view cameras could be remotely controlled 
for focus and zoom, but fixed views were used for all of the experimental tasks. 

EXPERIMENT GOALS 

This study is part of a longer-term effort to quantitatively evaluate a snapshot of 
present telemanipulation technology to expose areas in which improvements are needed 
for specific real-world applications. An additional goal was to explore the possibilities for 
the evaluation or preselection of operators of telerobots. As a first step in this direction, 
we looked at some well-known motor control tests to assess their predictive value for 
teleoperation performance. 

The overall experimental approach was to design a preliminary experiment which 
looked at a relatively large number of independent variables. The results of this study are 
being used to guide current work in which focused experiments are performed to obtain 
statistically powerful results. 

The tasks used for teleoperator evaluation fall naturally into two classes: generic tasks 
and application tasks. Generic tasks are idealized, simplified tasks which are designed to 
test specific telemanipulation capabilities. Application tasks are designed as much as 
possible to mimic real-world uses for teleoperation. Evaluations based on generic tasks 
illustrate the capabilities of existing telerobotic technology, while application tasks guide 
the technology in the direction of greatest payoff. 
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METHODS 


EXPERIMENT DESIGN 
Study Variables 

The experiment design varied control mode, task, and subject. The dependent mea- 
sures were task completion time, sum of squared forces (SOSF), and number of errors. 
Three repetitions of each subtask were performed by each subject in each of the control 
modes. All subtasks were performed in random sequence to form one repetition (random- 
ization without replacement). All subjects performed all repetitions of a given control 
mode before the next mode was tested. 

Robot sensing and control data were recorded by a Unix-based laboratory computer 
system through its real-time parallel port. This parallel port is connected to a port on 
the universal controller which can send any of the robot control parameters at the servo 
rate of 1 kHz. We recorded the six axes of force and torque information, as well as gripper 
opening position and gripping force sensed by the two fingers. A typical record of this 
raw data forms eight plots. Figure 4 shows one repetition of the peg-in-hole task. The 
information is serially multiplexed in a packet of 10 bytes for a net sampling rate of 100 Hz 
per channel. The complete data vector is described in Table 1. One of the nine data bytes 
is transferred every millisecond followed by a synchronization byte. Task information such 
as completion time and task performance strategy can be computed from the time records 
of forces and torques used in performing the task (Hannaford 1987). Data recording began 
and ended when the experimenter pressed a mouse button on the laboratory computer 
system. 

We measured force/torque information of the direct human operation by mounting 
the task modules on a force/torque sensor removed from the robot and attached to a stable 
base. The task board surface was mounted at the center of the force-sensing coordinate 
frame. This had the effect of reversing measured forces in the y and z directions relative to 
the robot tasks. The SOSF (see below) force performance measure ignores this difference. 
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X Axis Force V Axis Force Z Axis Force (lbs) 



Time (seconds) 


Figure 4. x, y, and z Forces during teleoperation. Raw data acquired from a single repetition 
of the peg-in-hole experiment (continued in Figures 5 and 6). x, y, and z axis forces measured 
in the robot-hand reference frame during insertion (r = 16 to t = 24 seconds) and extraction ( t = 
40 to t = 60 seconds) of the peg. Spikes represent taps made intentionally by the operator to 
provide reference benchmarks in the force signal. 
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Roll Torque Yaw Torque Pitch Torque (in.-lb) 



Time (seconds) 

Figure 5. Pitch, yaw and roll torques during the peg-in-hole task. Recording is from the 
same repetition as in Figure 4. 
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Gripping Force (lb) Jaw Opening 


Time (seconds) 



Figure 6. Jaw opening and finger gripping force for the peg-in-hole task. Jaw opening in arbi- 
trary units, gripping force (one trace for each finger) in pounds. Jaws open after insertion (r = 25 
seconds), close and open for the tap ( t = 27-28 seconds) and close again on the peg prior to ex- 
traction (t = 39 seconds). 
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TABLE 1 

DATA RECORDING CHANNELS 


# 

NAME 

RANGE 

SOURCE 

1 

Ja.w Position 

0-255 units 

Hand-position transducer 

2 

Finger Force 1 

0-100 lb 

Left-finger group force sensor 

3 

Finger Force 2 

0-100 lb 

Right-finger group force sensor 

4 

x Force 

0-10 lb 

6-axis wrist force-torque sensor 

5 

y Force 

0-10 lb 

6-axis wrist force-torque sensor 

6 

z Force 

0-10 lb 

6-axis wrist force-torque sensor 

7 

x Torque 

0-40 in. -lb 

6-axis wrist force-torque sensor 

8 

y Torque 

0-40 in. -lb 

6-axis wrist force-torque sensor 

9 

z Torque 

0-40 in. -lb 

6-axis wrist force-torque sensor 


Task Board 

The task board (Figure 7) consists of a 21-in. by 21-in. frame which accepts modules 
of either 7 in. by 7 in. or 14 in. by 7 in. The frame is adjustable in its vertical angle 
and can fold down completely for compact storage. The modular design allows for easy 
construction of new tasks. Modules can be moved or interchanged within the task board. 
All of our generic tasks have been constructed on the 7-in. by 7-in. module size. Detailed 
information on the task-board design is included in Appendices A and B. 

Another advantage of the modular task-board design is that the modules can be 
mounted individually on a six-axis force-torque sensor to enable force-torque recordings 
during direct manual operation of the tasks. This would be impractical without the modu- 
lar design because the large mass of a complete task board would cause enough gravitational 
force to saturate the sensor. 
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Figure 7. Two views of the task board and the robot hand performing the peg-in-hole task. 
The modular task board design has nine 7" x 7" openings for task modules. Shown are the four 
task modules used in the experiments: (a) Bayonet Connector and Peg-in-hole tasks (upper 
and lower left modules); (b) Velcro and Electrical Connectors tasks (upper and lower right mod- 
ules). Task modules could be removed and mounted on a separate force-torque sensor for mea- : 
surement of force and torques in hand operation. 
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Tasks 


The four tasks used were attach and detach velcro, peg insertion-extraction, manip- 
ulating three electrical connectors, and manipulating a bayonet connector. Each task in 
turn is broken down into component subtasks: 

Task 1 Velcro attachment. Exchange the position of two differently shaped blocks at- 
tached to the task-board module with velcro. Attempt to attach the blocks 
securely while minimizing unnecessary force. 

Task 2 Peg-in-hole matrix. This task consists of nine holes arranged in a square matrix. 

The rows each have a progressively larger clearance, and each column has a 
different chamfer. The subtasks are to take the standard peg and insert it into 
a given hole. The dimensions of this task were verified after performance of the 
experiments and are shown in Figure 8. 

Task 3 Electrical connectors. The subtasks consist of the mating and unmating of three 
standard electrical connectors: a three-prong chassis power-cord connector, a 
DB25 25-pin signal connector, and a 1/4-in. telephone-style plug. 

Task 4 Bayonet connector. This task consists of unlocking, unmating, mating, and lock- 
ing a Bendix bayonet-style electrical connector. 

Task Details 

Peg-in-hole is a popular task because it is easy to specify and thus can be, and has 
been, duplicated in many labs. In comparing peg-in-hole tolerances among studies in the 
literature, a useful measure is an information measure of precision, I: 

I = log, (H/(H - P)) 

where H is the hole diameter and P is the peg diameter. This measure is tabulated in 
Table 2 for some recent telemanipulation studies which used the peg-in-hole task. These 
dimensions can be placed in the context of industrial practice using Figure 9. 

TABLE 2 


STUDY 

HOLE SIZE 
(in.) 

PEG SIZE 
(in.) 

INFORMATION RANGE, I 
(bits) Figure 9 symbol 

Hill & Salisbury 1977 

2.000 

1.000-1.996 

1. 0-9.0 

A 

Hill 1979 

1.000 

0.250-0.984 

0.4-6. 0 

* 

McGovern 1974 

2.550-12.540 

2.540 

0. 3-8.0 

0 

Pepper & Kaomea 1988 

0.254 

0.295-0.386 

2. 0-5. 6 

o 

Present Study 

1.003-1.0006 

0.998 

6. 0-8. 6 

• 
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BOARD DIMENSIONS 



Figure 8. Dimensions of the holes and peg used in the peg-in-hole task. 
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DIA. (inch) 


100 


SOURCE: 

WHITNEY, D.E., "QUASI-STATIC ASSEMBLY OF COMPLIANTLY 
SUPPORTED RIGID PARTS," JOURNAL OF DYN. SYS. MEAS. & 
CONTROL, VOL. 104, P. 65, 1982. 


GENERAL MACHINE BEARING PRACTICE 

PRECISION LAPPED BEARING PRACTICE 

ELECTRIC MOTOR BEARING PRACTICE 


PEG IN HOLE TASK 
(PRESENT STUDY) 



0.0001 


0.001 


0.01 


CLEARANCE RATIO = ^ - 


Figure 9. Comparison of tolerances used in the peg-in-hole task with industrial design 
practice. 
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Note that this measure is different from similar “Fitts” measures used by Hill (1972) 
and Pepper et al. (1988). Our measure ignores the translation component and focuses 
instead on the difficulty of insertion and extraction of the peg once the tip is properly 
positioned. The Fitts measure incorporates the translation portion of the peg-in-hole task 
as: 

r _1 1 r 2A ) 

I F rp ( JJ p ) 

where A is the distance moved to reach the hole and T is the time to complete the movement 
and peg insertion. Our peg-in-hole task included about 6 in. of translation which was 
usually accomplished in about 10 sec. Thus, the maximum Fitts rate of our task was 
about 12 bits/sec. This type of analysis is more thoroughly reviewed by Wiker (1988). 

The other tasks used here are similarly easy to specify because they use commercially 
available components. The electrical connectors were unmodified except for a square grip- 
ping fixture attached to the locking ring of the Bendix multipin connector (type #PT06A- 
20-1GS/16P), and velcro is a commonly available material. 

Task Sequences 

Each task was performed according to a prespecified procedure for which the subjects 
(operators) trained. The task sequences were interspersed with “taps,” in which the oper- 
ator made momentary contact with a designated point on the task board either with the 
bare gripper or with a held object. The taps injected distinct spikes in the force record 
(especially the X axis, normal to the task-board surface, see Figure 4). All tasks began 
and ended with a tap on a designated square on the task-board surface. The resulting 
force spikes provided well-defined benchmarks for measurement and interpretation of the 
progress of the task by inspection of the force records alone. 

The tasks, subtasks, and sequences for the tasks are listed in Appendix C. 

Test Operators 

The test operators for this experiment were chosen from a population that would 
have technical background, but not an in-depth knowledge of robotic technology. Detailed 
robotic knowledge or knowledge of the specific system itself was felt to be distracting and 
to not reflect forseeable operator populations. Subjects recruited were graduate students, 
undergraduate students, and one technician, none of whom were specialists in robotics. 

Pretesting and Training 

Before telemanipulation training, the subjects performed two standard tests designed 
to assess motor-control ability. These tests were administered automatically by a graph- 
ics workstation. The motor control battery consisted of a rotary pursuit tracking task 
and a mirror tracing task. In the first task, the operator controlled a small green dot and 
attempted to keep contact between the dot and larger red dot moving in a circle at approx- 
imately 33 revolutions per minute. In the second, the operator drew the outline of a star 
between two path boundary lines in the presence of 0-, 90-, 180-, and 270-degree rotations 
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of cursor motion relative to hand motion. The tracking-task performance was measured by 
percentage of the time on target; the star-tracing task by percentage of line length within 
the displayed outlines. The subjects used a mouse to interact with the computer; the tasks 
were completely controlled and scored by the computer. 

After the “motor control” testing, each subject received 2 to 4 hours of practice on 
the telemanipulation apparatus. The practice sessions consisted of four 30-minute sessions 
in which the task set was performed with and without force feedback. 

ANALYSIS AND PERFORMANCE MEASURES 

The raw force- torque data are a rich source of information which can be understood in 
terms of the task description and which can in turn be used to quantify task performance. 
This section will describe computations which were performed on the force/torque data to 
produce performance measures. 

Data recorded at such a high rate (100 Hz) were computationally difficult to analyze 
and contained spurious information such as mechanical ringing signals showing up in the 
force/ torque plots (Figure 10). To simplify plotting and analysis, the data were filtered 
and reduced by a decimation process which averaged groups of 10 successive samples to 
produce a filtered output data file. Figure 11 shows the resulting data in which mechanical 
oscillations above 5 Hz are suppressed. 

Completion Time 

The simplest and most long-standing task performance measure, completion time, can 
be determined from the length of the data file containing the force- torque data. The length 
of the data file was determined by the time between the experimenter’s button presses. In 
most of the cases reported here the movement time was long compared to human reaction 
time and thus errors due to reaction-time variability of the experimenter are minimal. To 
prevent even this small error, our analysis software identified the starting and ending taps 
and used them as the basis of completion time. 

Sum of Squared Force (SOSF) 

This measure is computed by taking a nondecreasing sum of the square of the force 
or torque value 

N 

SOSF = J2 f> dt 

where N = number of data samples (task time over dt ), /; is the ?'th sample of force or 
torque, and dt is the sampling interval (0.01 sec in this experiment). SOSF is accumu- 
lated separately for each force and torque axis. An alternative and related force/torque 
performance measure is RMS force/torque. RMS force/torque is not preferred because it 
cannot be added among different movements, channels, or segments. Adding the RMS 
measure biases the resulting sum in favor of shorter segments or tasks. As with completion 
time, SOSF can be computed over the whole record or between starting and ending taps. 
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Figure 10. Unprocessed data from the force-torque sensor. Note expanded time scale. Data 
was filtered by averaging adjacent non-overlapping windows of 10 samples (see Figure 11). 
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Figure 11. Filtered data. Same data as in Figure 10, but filtered with 10-sample averaging win- 
dow. 
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Visual Error Scoring 

A third way in which performance can be measured is through an observer’s notations 
of the “quality” with which a task is performed. In our experiments, a set of “errors” 
was defined and explained to the test operators and experimenters (see Table 3). An 
experimenter watched each repetition of the experiment and counted occurrences of each 
error. 


TABLE 3 
Task Errors 


SYMBOL NAME DEFINITION 


S 

I 

GI 

D 

E 


Slip 

Incomplete 
Gripper Interference 
Drop 

Excessive Force 


Object slips in jaws. 

An inserted object such as a peg is not completely 
inserted. 

Gripper touches an object not related to the current 
task. 

Gripped object is dropped. 

Visible deflection of the task board. 


Individual differences among error scorers might be a source of bias in this measure. 
To detect this possible bias, four of the experimenters scored the same experimental run. 
Their error counts were substantially in agreement. 

Task Segmentation Analysis 

In some cases it is desirable to compare performance measures among different seg- 
ments of the same task. For example, to compare the completion time and SOSF for peg 
insertion with peg extraction. This was accomplished through a computer program which 
could recognize benchmarks in the force signal, and divide it in time between a set of seg- 
ments. Returning to the peg-in-hole example (Figure 12a, b, c, see especially “x force” in 
part a), the data can be clearly divided into “translation” (the manipulator is in free mo- 
tion; no contact forces), “taps” (sharp spikes in force), “insertion” (predominantly positive 
forces), and “extraction” (predominantly negative forces). Both “completion time” and 
SOSF can thus be computed for each segment of the task. This could potentially identify 
which task segments are more or less affected by a particular system improvement. 

The segmented analysis program uses a description of the expected task sequence 
entered by the operator. This description is entered in a simple language expressing a 
rough model of the expected forces developed during performance of a task. The elements 
of this language are single letters describing typical force patterns (see Table 4). Referring 
to the task description of the peg-in-hole task for example, the experimenter can write the 
expected force sequence as: 

“m,t,m,i,m,t,m,x,m,t,m” 
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Time (seconds) 


Figure 12. Segmentation of task performance data: shown is an example of x axis force data 
from the peg-in-hole task automatically divided into phases by the segmentation algorithm. 
Designation of phases is: 

m Move: translation phases of the task involving no contact forces. 
t Tap: momentary intentional impact with task board surface — a timing mark. 

/ Insert: forces arising from insertion of the peg. 

X Extract: forces arising from extraction of the peg. 
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Table 4 


Task Phase Description Language 

Symbol Definition 

m Movement: free motion of the slave, contact forces are negligible, 

t Tap: a sharp force impulse. 

i Insertion: sustained predominantly positive force, 

x Extraction: sustained predominantly negative force. 


Segmentation Algorithm 

The segmented analysis program uses this description to drive a thresholding state ma- 
chine which identifies the time at each transition. This is accomplished by first identifying 
the taps in the data. 

Each tap separates the task into smaller actions. Before and after each tap is a 
short period of time when no forces are exerted on the hand. This enables us to separate 
the taps from the rest of the data. The algorithm finds the taps in the x-force data so 
that completion time and SOSF analysis can be done on the individual actions. A tap is 
defined by several different parameters including minimum height, h, maximum width, u>, 
minimum time before the tap when no forces occur, tb , minimum time after the tap when 
no forces occur, te, tolerance to allow for noise during the periods before and after the 
tap, tol , and finally the maximum time between the beginning of the tap (when the force 
crosses the tolerance level) and when the force is greater than or equal to the minimum 
height, t 2 1 (Figure 13). 

The tap algorithm is split up into four states. In each of these four states, specific 
conditions must be met to cause a transition to the next state; if not, the algorithm jumps 
back to state 1 and analysis continues until the end of the data. 

State 1 Move forward through the x-force data until the end of the data or until crossing 
the minimum height (force) threshold, h. Mark this position. 

State 2 Jump backward through the x-force data the minimum length of time for a tran- 
sition before the tap, tb. Step forward in the x-force data until the force exceeds 
the tolerance tol. Mark this position. Check to see if the length in time be- 
tween this mark and the mark from state 1 is within the limit defined by the tap 
definition, t 2 i . If not, go back to state 1; if so, proceed to state 3. 

State 3 Jump forward in the x-force data from the state 1 mark to the width of a tap, w, 
plus the minimum length of time for a transition, te. Step backwards until the 
force jumps outside of the tolerance, tol , and mark the position. Check to see if 
the length in time between this point and the state 2 marker (the beginning of 
the tap) are within the definition of a tap width, w. If not, go back to state 1; if 
so, proceed to state 4. 
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X Axis Force 



h = Minimum Tap Force 

w = Maximum Tap Width (time) 

tb = Minimum Contact-free time before tap 

t e = Minimum Contact-free time before tap 

tol = Noise Threshold 

t 2 i = Front Porch Time 


© State 1 
© State 2 
© State 3 


Figure 13. Diagram of algorithm for identification of a "tap," a voluntary impact designed into 
the task definition to act as an event marker. The algorithm is a state machine driven by thresh- 
old crossing events. The six parameters define the permissible variation in tap height, width, etc. 
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State 4 A tap has been found. Store the beginning and end of the tap and calculate the 
center of the tap. Return to state 1. 

After the taps are located, the movement time is calculated, and boundaries are 
defined. for the insertion and extraction phases. Completion time and SOSF can then be 
accumulated for each task phase. 
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RESULTS 


SUMMARY RESULTS 

The results of the experiments form a complex set of data points on the three-way 
experimental design. Each point in the three-dimensional space formed by the control 
modes, subjects, and tasks contains three reduced-data points for each of the three rep- 
etitions of each experimental trial for each of the five subjects. These are the completion 
time, SOSF, and number of errors. 

The reduced data can be further simplified by averaging across one or more dimensions 
of the design. As a first look at the data, we have computed averages over all subjects 
and over the first three tasks, “velcro,” “peg-in-hole,” and “electrical connectors.” The 
fourth task was not included in this average because it took significantly longer than the 
others (approximately 150 seconds vs. 75 seconds), and was often not completed due to its 
difficulty. There are nine subtasks for the peg-in-hole task (corresponding to the nine test 
holes) vs. two for the velcro and three for the electrical connectors. These averages (Figures 
14, 15, and 16) show clear trends in performance as the level of capability progresses from 
position control, through force reflection, up to the bare-handed operator. 

Completion time (Figure 14) for the three primary tasks drops from an average of 92 
seconds with position control to an average of 63 seconds when force feedback is added. 
Completion time drops to only 14 seconds for the bare-handed human. SOSF (Figure 15) 
drops even more dramatically (from about 3500 to 500 lb 2 -sec) when pure position control 
is augmented with force feedback and further (to 200 lb 2 -sec) for the bare-handed case. 

The number of errors observed per repetition drops from 3.0 to 1 . 1 when force feedback 
is added. No errors were observed in the bare-handed data. 

The difference between the first two cases and the bare-handed case is so great as to 
not require statistical validation. The two-tailed Z test was used to test the null hypothesis 
that the differences between position control and force-feedback control performance were 
due to chance. The number of subtask repetitions performed under each control mode 
is shown in Figure 17. Because the number of data points is large compared to one 
(approximately 130 to 150), the Z statistic could be used. 

The probability of the null hypothesis calculated by this procedure was much less 
than 0.01 in all of the differences reported above, giving them a high degree of statistical 
significance. 

These results summarize one of the main results of this report, that the provision of 
force feedback reduces completion time for a task mix emphasizing energetic interaction 
and precision manipulation by approximately 30 percent, reduces SOSF by a factor of 7, 
and reduces errors in performing the task by 63 percent. 
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Figure 14. Average completion time performance for five operators performing a mix of tasks 
under three conditions: (1) position control only, (2) position control with kinesthetic force feed- 
back, (3) direct operation with the bare human hand. 
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Figure 15. Sum of Squared Forces (SOSF) performance measure for the same operators and 
tasks as in the previous figure. SOSF is a measure of the total magnitude of force required to 
perform the task. SOSF drops dramatically as manipulation performance is improved. 
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Figure 16. Average number of errors made by the operator in each 
The errors were predefined and understood by the test operators. 


repetition of each task. 
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Number of Data Points 


Total Number of Data Points 

for all Tasks and Subjects 



Configuration/Mode 


Figure 17. Number of data points for all of the experiments. Each data point represents one 
subject doing a single performance of one of the subtasks. Visual, mixed, and hybrid modes were 
studied as preliminary experiments with fewer repetitions and subjects. 
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The motor control pretesting was performed in order to determine whether the perfor- 
mance of individual operators in teleoperation could be predicted from their performance 
on the pretests. A summary of these results is presented below. For further details, see 
Appendix D. 

The analysis attempted to compare performance on the various motor control tests 
with performance in the various telemanipulation tasks. Linear regression was computed on 
the points formed by plotting the performance of each operator on selected axes consisting 
of one pretest performance measure and one measure of telemanipulation task performance. 
The approach was to check for statistically significant correlations which would indicate 
the predictive power of the selected tests. The significance of a particular correlation is 
determined by the permissible P value (in this case 0.05) and the computed value of the 
correlation coefficient r. For a given number of data points (subjects) there is a threshold 
on r below which the correlation is not. significant. The results were that the number of 
significant correlations was approximately equal to that expected by chance. Thus the 
selected motor control tests appear not to predict telemanipulation performance. 

Because of the small number of subjects, only very strong correlations can be detected 
with high confidence. Repetition of similar experiments with more subjects is underway 
to increase the ability to detect smaller but still significant correlations. 

DETAILED RESULTS 

As an example of the raw data resulting from these experiments, the complete, filtered 
data set for a single repetition of the peg-in-hole task shows a complex coupled set of time 
functions (Figures 4, 5, 7) in which forces and torques vary (Figures 4, 5) as the jaws release 
and then regrasp the peg (Figure 6, t = 25 to 40 seconds). Recall that the task is defined 
to start and finish with the peg held in the gripper. The x-axis force (Figure 4, bottom 
trace) in conjunction with the jaw opening (Figure 6, top trace) provides a clear picture of 
the task sequence. The operator starts by tapping the grasped peg against the task board 
and moves to the hole. A large positive x-force indicates the struggle to insert the peg, 
which is complete when the jaws release it at t = 25 sec. The jaws reclose on empty space 
to tap the back of the inserted peg ( t = 32 sec), and the peg is regrasped at t = 39 sec at 
which point withdrawal almost immediately begins. After the peg is completely removed 
and contact forces cease ( t = 60 sec), there is a short translation to the mark, and a final 
tap. Similar sample records from the other tasks are reproduced in Appendix E. 

When selected experimental records from the several control modes are compared 
(Figure 18), the x-axis force traces tell most of the story for the peg-in-hole task because 
of the alignment between the task axis and the force/torque sensor’s x-axis. Comparison 
of performance in the several control modes shows the reduced completion times and force 
levels achieved as more capability is added to the system. 

In the visual feedback condition (Figure 18) the operator uses position control without 
kinesthetic force feedback, but receives contact force/torque information by means of the 
graphic display. With only the visual force/torque information, the task is completed in 
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Figure 18. Illustration of force record changes with control modes. Individual records of X ax- 
is force for 4 repetitions of the peg-in-hole task under different control modes, (a) manual, (b) 
enhanced control; "Hybrid Force Accommodation", (c) force feedback, (d) visual feedback. 
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about 100 seconds, and peak forces of up to +10 lb and -9.5 lb are observed in the insertion 
and extraction phases respectively. When kinesthetic force feedback is used (Figure 18b), 
completion time drops to 82 seconds, and peak forces drop to approximately ± 5 lb. 

Sensed force and torque information can be used locally by the manipulator control 
system to implement an effective compliance or, in general, a mechanical impedance of 
the manipulator. Performance with one example of this type of shared control is shown in 
Figure 18(c). In this mode, the orientation axes were controlled automatically to move in 
the direction of sensed torques. Position axes were controlled by the operator. Completion 
time was reduced to 60 seconds. Forces were substantially reduced only for the extraction 
phase, in which the peak force was —4 lb. 

Finally, when the task was performed manually (Figure 18d), the task-related forces 
were practically invisible compared to the taps. Completion time was about 15 seconds, 
and peak forces reached +3 lb and —0.5 lbs. 

Although fascinating in themselves, these raw data records are isolated anecdotes of 
individual task performances. As described in the summary above, data from all repetitions 
of the experiment were reduced to the three basic performance measures. The visually 
scored error rates were manually collected with the reduced performance data. 

The first level of detail added to the summary results was to break them down by 
individual task instead of averaging all the tasks together (Figures 18a, b, c). Doing 
this showed that the effect of force feedback was not the same for all of the tasks. For 
the peg-in-hole task, completion time followed the expected course, dropping by almost 
a factor of two from 105 to 59 seconds as force reflection was added. For the velcro- 
blocks task, completion time increased from 72 to 83 seconds. Both of these changes were 
statistically significant when analyzed by the method described above. For the electrical 
connectors, when force feedback was added only a slight change from the completion time 
of 70 seconds was observed which was NOT statistically significant. Of course, all of the 
tasks were completed much faster by the bare- handed operator. The average time in this 
case was about 15 seconds. 

The SOSF data tell a different story. As with completion time, for the peg-in-hole 
task there was a dramatic drop in SOSF (from 5400 to 500 lb 2 -sec) as force reflection was 
added. The increase in completion time seen for the velcro task was accompanied by a 
significant decrease in SOSF (from 800 to 400 lb 2 -sec). For the electrical connectors, the 
SOSF measure declined significantly despite the unchanged completion time. 

The data of Figure 19 showed the variation of performance with control mode as 
task was varied. The same data could be replotted to highlight the task dependence of 
the effects of mode (Figures 21, 22). The first task (positioning velcro blocks) showed no 
significant effect of force feedback on completion time (see above). The SOSF (Figure 22) 
was reduced by about 1/2 to a value close to the manual level. The second task, peg-in- 
hole, showed a significant reduction in both completion time and SOSF with the addition 
of force feedback. 
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Figure 19. Completion time (CT) as a function of control mode broken down by the individual 
task type: velcro blocks (solid line, circles), peg-in-hole (dotted line, squares) and electrical 
connectors (dashed line, triangles). Force feedback affects each task differently. The changes 
are all statistically significant except the slight drop in CT for electrical connectors with force 
feedback. Note that force feedback significantly increased CT for the velcro task. 
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Figure 20. Sum of squared forces (SOSF) measure for the three task types. Key same as for 
Figure 19. SOSF dropped significantly for all of the tasks when force feedback was used. Lev- 
els with force feedback were still approximately double those recorded from the bare-handed op- 
erator. 
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Tasks: 

Task 1 : Velcro Blocks Task 3: Electrical Connectors 

Task 2: Peg in Hole Task 4: Bayonet Connector 


Figure 21. Completion time plotted for each mode as a function of task. This is the same da- 
ta as Figure 19, but replotted with the addition of data from the bayonet connector task. Diverg- 
ing traces highlight the task dependence on the differential effect of force feedback. 


37 







The third task, electrical connectors, showed no change in completion time, but a 
significant drop in SOSF for all three connector types when force feedback was added. 
The last task, the bayonet connector, was similar in that there was a smaller change in 
completion time than in SOSF (see the last columns of Figures 23 and 24). The relative 
changes in completion time and SOSF with force feedback thus appeared to depend on the 
characteristics of the task. 

Within an individual task, the results could be further broken down by subtask. This 
was done for the nine holes of the peg-in-hole matrix (Figures 23 and 24) (recall that the 
holes varied in diameter and in degree of chamfer). Referring to Figure 8 for dimensions, we 
see that diameter is constant for holes 1-3, 4-6, and 7-9. Thus a dependence of difficulty 
on hole diameter would be expected to show up as a stepwise decrease in performance 
(increase in completion time (CT) or SOSF) with hole number. Chamfer varied along the 
rows of the matrix. Thus, dependence of performance on chamfer would be expected to 
show as a sawtooth dependence of CT and SOSF on the hole number modulo 3. 

Neither of these dependencies was conclusively demonstrated by either the comple- 
tion time or SOSF data. Thus, for the range of information measures covered by our 
experiment (6 to 8.6 bits), performance was remarkably independent of hole diameter or 
chamfer. Furthermore, the performance without force reflection was uncorrelated with the 
performance with force reflection. 

When the electrical connectors task was broken down by subtask, we measured per- 
formance for the individual connector types (Figures 25 and 26). Completion time (Figure 
25a) varied from 60 seconds (1/4-in. and 3-prong connectors) to 100 seconds (25-pin 
Cannon D connector). Improvement in completion time was slight or nonexistent with 
force feedback (triangles, dashed line). The force measure, however, showed substantial 
improvement for all three connectors. The force measure ranged from 800 to 2700 lb 2 -sec 
for the connectors without force feedback to 200 to 700 lb 2 -sec with force feedback. The 
improvement was greatest for the 25-pin D connector. Note that for these tasks, substan- 
tial improvement was shown in SOSF performance while there was little or no change in 
completion time. 

The CT and SOSF performance measures are well defined not only for the whole 
task, but for the individual segments of the task as well. The segmentation algorithm 
described above was used on 150 repetitions of the peg-in-hole task. The results were 
completion times and SOSF measures for the individual segments: “movement,” the free 
motion of the peg between hole and tap location; “insertion,” the phase of the task from 
tip contact to grip release; “extraction,” from gripper-peg contact to tip clearance; and 
“tap,” momentary peg-to-board contact. 

The CT was plotted for each of the segments comparing force-feedback and position- 
control modes (all subjects, Figure 27). As expected, completion time for the movement 
and tap phases were unchanged by the addition of force feedback (at about 32 seconds and 
2 seconds respectively). The insertion and extraction phases were accomplished markedly 
faster (11 vs. 31 seconds for insertion, 10 vs. 35 seconds for extraction) when force feedback 
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Subtasks 


Figure 23. Completion time vs. hole number for the different tolerances in the peg-in-hole 
task. Hole geometry is defined in Figure 8. Dependence on hole diameter would show up as a 
stepwise increase in CT with jumps at #4 and #7. Dependence on hole chamfer would show up 
as a sawtooth dependence of CT on hole number modulo 3. Neither of these dependencies is 
conclusively demonstrated. Dependence would most likely show up if a wider range of hole tol- 
erances had been used. 
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Figure 24. Dependence of SOSF on hole geometry for the peg-in-hole task. Results are sim- 
ilarly negative for the range of hole sizes (relatively high precision, see Figure 9) used. 
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Subtasks 


Figure 25. Completion time dependence on the connector type. Key to Subtasks: (1) 1/4— in. 
plug, (2) 25-pin DB25 RS232 connector, and (3) 3-prong chassis power connector. For these 
tasks, force feedback made no significant improvement in completion time. 




Subtasks 


Figure 26. Force performance by connector type. SOSF is substantially reduced for all three 
connectors (see Figure 25 legend for key). The connector task appears to involve mostly preci- 
sion alignment of the components, which is not speeded-up by force feedback. However, spuri- 
ous forces, which are not required to mate the connectors but are generated in the course of ex- 
ploratory manipulation, are minimized. 
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Figure 27. Changes in completion time by task phase for the peg-in-hole task. Force feed- 
back substantially reduces the time to perform insertion and extraction without significantly af- 
fecting the time spent in translation or tapping. 
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was present. Thus force feedback differentially affected different task phases. 

The differential effect of force feedback dramatically changed the proportion of the 
total time spent in each of the task phases. Pie charts illustrate the mix of manipulation 
types in the task. Without force feedback (Figure 28) the task was dominated by the 
insertion and extraction phases: the task components involving energetic interaction with 
the environment. Together they accounted for 2/3 (67 percent) of the total task time of 
102 seconds. When force feedback was present (Figure 29) the total time was reduced to 
55 seconds, and the proportion of time spent in energetic interaction was reduced to 37 
percent. In contrast to the cases without force feedback, the dominant proportion of the 
total time with force feedback was consumed by the free-motion portions of the task. 

Another variable of interest is the individual operator. Are there differences in per- 
formance between the different subjects? If so, are they consistent and predictable? Do 
different operators appear to follow distinct and identifiable strategies? Although this 
experiment was not designed to be a statistically rigorous study of these effects, certain 
regularities have emerged which inform speculation on these questions. 

The task performance measures can be broken down and individually computed for 
each subject (Figures 30, 31, and 32). On the basis of this limited sample, completion time 
(Figure 30) for the first three tasks was approximately the same (approximately 105 sec) 
for all five subjects except for subject 5 who was consistently faster (approximately 50 sec). 
The SOSF varied with the task but varied relatively little (i.e., by an amount comparable to 
one within-subject standard deviation) among subjects (Figure 31). Standard deviations 
in completion time appeared to be relatively invariant at about 30 seconds across the 
subjects for tasks 1-3. Standard deviation (s.d.) in completion time was also relatively 
constant at about 105 seconds for the difficult bayonet connector task. Error rates (Figure 
32) varied markedly across the subjects as did the standard deviation of error rate from a 
high of 3 errors per repetition (s.d. = 2.5) to a low of 1 error per repetition (s.d. = 0.5). 
Subject 5 had the lowest (best) value in each of the three performance measures for most 
tasks. 

PRELIMINARY RESULTS: ADDITIONAL CONTROL MODES 

Future applications of telemanipulations will rely increasingly on the sharing of control 
between the human operator and, at first, limited autonomous functions. Beyond the 
three basic control modes reported above, Position Control, Kinesthetic Force Feedback, 
and Direct Manual, three additional modes (defined below) were evaluated in smaller 
experiments. These experiments were of smaller scale because of limitations of time and 
because some data were lost due to electromagnetic interference problems. (The number of 
data points for each of the six modes is shown in Figure 17.) The generality of these results 
is thus somewhat limited. However, they are reported because they still represent three 
repetitions of each subtask element, and are our first source of quantitative performance 
information on these advanced modes. 
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Time for Segmented Task using No FFB 

Task #2: Peg in Hole 
Total Time: 105 sec 



Figure 28. Task phase breakdown with Position Control. Without force feedback, total com- 
pletion time is approximately evenly divided between moving, insertion, and extraction in the 
peg-in-hole task. The data are the same as the dotted line of Figure 27. 
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Total Time: 55 sec 



Move (59.4%) 


Figure 29. Task phase breakdown with KFF. When the task is performed with force feed- 
back, total time drops from 105 sec to 55 sec. As a result, the completion time is dominated by 
the translation phases of the task. Changing the level of manipulation capability redefines the 
task weights. 
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Figure 32. Error rate showed the greatest variation among the test operators. 
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The three additional modes were repeated three times each for the complete task set 
by at least one subject. One relatively primitive mode, “Position Control with Visual 
Force/Torque Display,” and two modes of shared control, “Shared Force Accommodation” 
and “Hybrid Force Accommodation,” were tested by at least one subject. These modes 
are defined above in the “Capabilities” section. 

Two subjects whose performance was quite similar in the modes for which both data 
sets exist were compared in all six modes for all of the first three tasks. Completion 
time performance (Figure 34) is slightly improved (83 seconds vs. 100 seconds) with the 
use of the graphical force display (significance not tested) although not by as much as 
kinesthetic force feedback (mode 3). “Shared Force Accommodation” mode resulted in 
the lowest overall completion time of any of the telemanipulated modes (approximately 
44 seconds). “Hybrid Force Accommodation” in which both subjects had approximately 
equal performance, showed an increase in completion time compared to kinesthetic force 
feedback from about 70 to 80 seconds (significance not tested). SOSF performance (Figure 
35) showed larger differences in the same pattern. 

The third performance measure, task error rate (Figure 36) varied in a similar pattern 
(no data were available for the hybrid mode). In this plot, data from each subject were 
averaged for each point. Thus, for mode 1, data from all subjects are included, whereas 
for modes 2, 4, and 5 (see Figure 36), only one subject is represented. For completeness, 
completion time and SOSF are broken down by task type (Figures 37 and 38). These data 
are averaged as described for task error rate. 
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Figure 33. Video display of force-torque information. The bar graphs form a pseudo perspec- 
tive view of the hand coordinate frame. The diagonal bar represents the X direction (in and out 
of screen). Bars at upper, right, and lower edges represent the moments around the X, Y, and Z 
axes respectively. Bars on right represent finger opening and clamping forces. 
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Figure 34. Completion time for additional modes. Preliminary experiments were performed 
with three additional modes: visual force display, and two shared control modes, "Shared Force 
Accommodation" and "Hybrid Force Accommodation" (see text). Completion time with 
"Shared Force Accommodation" was the best of any telerobotic mode tested. Generally speak- 
ing, performance improved as the level of capability increased with the exception of "Hybrid 
Force Accommodation" mode, which did not improve performance. Because of the limited 
amount of data, these results are not statistically conclusive. 


53 




Force (lb 2 -sec) 


5000 

4500 

4000 

3500 

3000 

2500 

2000 

1500 

1000 

500 

0 


Figure 35. SOSF with additional control modes. The force control performance shows larger 
changes in the same general pattern as completion time. 
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Number of Errors 



Modes 


Figure 36. Number of task errors. Visual force feedback (mode 2) has a larger effect on er- 
ror rate than CT or SOSF. Data for mode 5 are missing. Bare-handed human operator ("mode" 
6) made no errors. 
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Modes 


Figure 38. SOSF performance as a function of control mode. 



DISCUSSION 


This study has characterized an advanced-telemanipulation-system prototype in quan- 
titative terms relevant to application planning. As yet, only recent theoretical develop- 
ments in the modeling of bilateral manipulation systems (Hannaford 1980, Raju 1988) can 
provide a full characterization of the system’s performance independent of the operator, 
and it remains to relate these performance measures to performance on actual tasks. Thus 
the approach was to use multiple test operators performing an array of different tasks 
under the various operating conditions. 

SYSTEM AND CONTROL MODES 

One of the unique features of the ESAB system is its astonishing number of possible 
control modes. Since each of the six task-space axes can be set into one of ten modes 
independently, one million combinations are generated. In addition, several parameter 
values must be specified for each axis. So far we have used only simple heuristics to choose 
among the huge number of ways the system can be controlled. The preliminary evaluation 
of two of these possible modes (Figures 34 through 36) showed that “Shared Force Ac- 
commodation” — a shared control mode in which Kinesthetic Force Feedback Control was 
used in translation and rotation axes and Local Force Accommodation used in orientation 
axes — resulted in improved performance (relative to force-reflecting teleoperation), and 
that “Hybrid Force Accommodation” — in which pitch and yaw axes were locally con- 
trolled and translation was operator controlled without force feedback — resulted in poor 
performance. This demonstrates that emerging forms of shared telerobotic control must 
be carefully selected and matched to the task. With the present state of knowledge of 
manipulation, we have no way of knowing at this point whether among the thousands of 
other possible modes there is one which enables even better performance than observed 
under “Shared Force Accommodation.” 

It is reasonable that visual feedback of force/torque information should result in a 
performance level intermediate between Position Control and Force Feedback (Figures 
34 through 36). However, an earlier study (Hannaford 1987) did not find improvement. 
Among the differences between the studies which might explain this are the use of rate 
control in the earlier study vs. position control here, better procedures and training for 
the test operators, and the improved dynamics of the mechanism and control system used 
in the current study. 

MECHANICS AND STABILITY 

One of the key performance issues in design of active telemanipulators is stability. The 
stability issue arises when forward position commands and kinesthetic force feedback com- 
bine to form a closed-loop system encompassing the human operator, hand controller, robot 
manipulator, and environment. Various studies (Hannaford k Anderson 1987, Hannaford 
k Fiorini 1988, Anderson k Spong 1988, Hannaford 1989) have put forward analytical, 
computational, and experimental models of this system and its dynamics. This work was 
consistent with operational observations that stability during manipulator contact was in- 
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versely related to force- feedback scale. This parameter, the force-feedback scale factor, is 
referred to as h\2 in the hybrid two-port nomenclature described in Hannaford (1989). 
This scale factor could be reduced to obtain stable operation. In the experiments reported 
here, h\2 was 0.1 so that only 1 lbf was applied to the operator for each 10 lbf applied to 
the manipulator tip. However, the cost of the stability obtained in this manner was not 
only reduction of the kinesthetic feedback of force information, but also, effective magni- 
fication of the force perception threshold of the man-machine system. This is so because 
the mechanism of the force-reflecting hand controller has significant friction levels between 
the actuators and the handgrip which effectively absorb all force signals below a certain 
force level, Fmin. The human operator has some perceptual threshold for forces, Fth. 
Thus, the man-machine perception threshold is 

_ (F min+Fth) 

Tmm - M2 

in the sense that any force applied to the slave will not be felt if its magnitude is below 
Tmm . In our case, the effective man-machine threshold is then 10(jPmin +Fth). For 
mechanisms available today, F min can be on the order of 8 oz., which in the above 
example, neglecting the human operator threshold, scales to an effective deadband to force 
sensitivity of 5 lb. Reducing this threshold is a major challenge in telemanipulation system 
design. 

TASKS 

The tasks studied fell into one of two broad classes, “generic” and “application” tasks. 

Although the main generic task (peg-in-hole) is a classical task in the robotics lab, 
its relationship to the “real world” is illustrated by the relationship of the tolerances used 
in our task board to industrial parts-mating practice (Figure 9). The peg and hole sizes 
reported here as well as those used by Hill (1977) (open triangle in the figure) are squarely 
in the range of “General Machine Bearing Practice.” One difference between the peg-in- 
hole task and the industrial parts illustrated is that the pegs are long compared to their 
diameter and thus have large contact surfaces. As a result, near the point where the tip of 
the peg has just entered the hole, the bending moment can be quite large, causing binding 
problems at that point in the task (as well as near extraction). Thus the parts-mating 
capability demonstrated in this and other studies may be extendible to tighter tolerances 
for parts with lower ratios of length to diameter. 

The electrical connectors qualify as application tasks because they are stock items 
unmodified for the experiments. Despite the occasionally high contact forces encountered 
in some of the experimental conditions and the large number of repetitions, none of these 
stock electrical connectors suffered damage from the robotic operations. In all of the con- 
nector tasks, the task board contained the female connector and the male was manipulated 
by the robot. 

In both the velcro and electrical connector tasks, and to a lesser extent in the bay- 
onet connector task, kinesthetic force feedback substantially reduced SOSF but did not 
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substantially affect completion time. This indicated that these tasks are primarily precise 
positioning tasks and that while force feedback did not aid task performance in terms 
of time, extraneous forces arising from contact with the task board were reduced by the 
kinesthetic force feedback. 

SUBJECTS 

Variation among subjects was surprisingly slight. Their backgrounds were similar 
(engineering students or recent graduates) except for one who was a physical education 
major with training in gymnastics and coaching. This subject showed the best overall 
performance by each of the measures. This apparent correlation between performance and 
prior academic background might suggest that potential operators be grouped into classes 
based on interests or aptitudes and that these groupings might predict relative levels of 
telemanipulation performance. Of course this must be tested with statistically significant 
numbers of test operators. 

We cannot conclude that there was any predictive value to the “motor control” tests 
the subjects performed prior to the tclemanpulation experiments. Although further exper- 
imentation is under way in this area, new preliminary results tend to confirm this negative 
finding. Of course, some other test may be found which will reliably predict telemanipu- 
lation performance. The personality- type hypothesis described above needs to be further 
tested. If it appears to hold, then surveys or personality inventory tests might prove useful. 

CONCLUSIONS AND RECOMMENDATIONS 

The main results of this study are that manipulation performance can be quantified 
along at least three dimensions, and that these measures showed task dependent increases 
in performance as system capability was increased. The unaided human operator was 
placed at the highest extreme of the scale of manipulation capability because present- 
day remote-manipulation systems all introduce substantial distortions and movement con- 
straints between the operator and task, and thereby reduce performance. It is expected to 
take much time before telemanipulation performance will equal that of the bare-handed hu- 
man. This study has quantified the ground remaining to be covered. In striving to achieve 
this level of performance, these results should be interpreted as data supporting more 
advanced modes of telemanipulation than are presently in widespread use. Although addi- 
tion of kinesthetic force feedback is of substantial help in moving performance towards the 
extreme demonstrated by the bare-handed human, more advanced shared-control modes 
offer further improvements. It is also likely that further improvements in the fidelity of 
kinesthetic force feedback will substantially improve performance. 

As was found in the studies mentioned in the introduction, this work supports the 
idea that multiple measures of performance must be used to characterize telemanipulation. 
This is illustrated by cases such as the electrical connector tasks, in which kinesthetic force 
feedback did not substantially affect completion time, but did sharply reduce extraneous 
forces. In this case the precision positioning demands of the task were not aided by force 
feedback, but potentially damaging forces could not be controlled without it. 


60 



FUTURE WORK 


Experimentation using similar tasks and methodology in which time delay is added 
between master and slave sides is already under way. These experiments will be useful 
for systems applications such as the Space Station, in which interfacing constraints add 
time delays on the order of 50 msec, as well as future applications such as ground-based 
teleoperation in low earth orbit, in which delays from 2 to 4 seconds will be involved. 

Other plans include expansion to dual arm capability and more complex tasks such as 
module exchange and assembly. The direct human control data should be supplemented 
with data from subjects wearing pressurized space gloves and calibrated against experi- 
mental EVA. 
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Appendix A 

Modular Task Board For Telemanipulation Experiments 

Douglas McAffee 
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MODULAR TASK BOARD FOR ROBOTICS 
AND TELEOPERATORS 


The Modular Task Board, depicted in the following pages, was 
developed for testing human subjects in experiments using the 
Advanced Teleoperator System. 

The Modular Task Board is composed of as many as nine separate 
"task windows" designed to simulate generic and actual tasks that a 
teleoperator /telerobotic system may be called upon to perform. 

Main Features 


Task Layout Easily Rearranged 

The distances between tasks can be changed. Sometimes it is 
desirable to vary the relative location and inter-task distances 
to offset human learning, check adaptation, or provide variety. 
Also, inter-task distances can be easily changed if after use 
modification proves desirable. 

Accommodates a Variety of Task Sizes 

The task board features removable window frames so that 
tasks wider than the standard 7-in. by 7-in. window can be 
accommodated. The task frame itself is modular and allows 
for larger tasks (up to 7 in. by 21 in.) to be mounted flush to the 
task board surface by removing intermediary frame members. 

Task Modification Easy and Inexpensive 

New tasks can be easily fabricated and added without 
removing or remaking the entire task board. If a task proves to 
be undesirable or becomes obsolete, the whole task board does 
not have to be scraped. 

No Down Time 

New tasks, modified tasks, or repairs can be made off-line to 
task board usage, thereby allowing for continuous system 
operation. 
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Task Isolation 


Often multiple task boards consist of a complicated, congested 
array of multiple tasks, or, to avoid overcrowding, they limit 
the number of separate tasks that can be performed. 

Sometimes it is desirable to test only one or two tasks 
sequentially and to focus on them only. The Modular Task 
Board allows the user to isolate individual tasks by filling the 
balance of the task board surface with "blank task windows." 

Even Task Board Profile 

The Modular Task Board is designed to allow most tasks to 
mount almost flush to the task board surface. This helps to 
avoid complicated task board topologies that could distract 
from the experiments being performed. 

Adjustable Inclination 

The task board inclination can be adjusted to any angle 
between horizontal and vertical. 

Optimal Use of the Puma 560 Work Envelope 

The task board is designed to fit optimally in the Puma 560's 
work envelope. When positioned properly, the entire task 
board surface can be reached. 
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TASKS CAN BE ISOLATED BY 
USING BLANK WINDOWS 



70 





Tasks Can Vary Bn Sizs 


Tool Box 

CCD cm CCD 

1/4"Scoket Drive Phillips Probe 



nn cm cm 



Full Size Electronics Card Cage 



D-SUB 


PEG-IN-HOLE MATRIX 

• • • 

o © © 












Modular Task Board 
Moimtine ConfigErattioms 


72 





MODULAR TASK BOARD and PUMA 560 ROBOT ARM 
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Task Board Modifications and Improvements 
Brent Guggisberg 
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INTEROFFICE MEMORANDUM 


19 December 1988 

TO: Blake Hannaford 

FROM: Brent Guggisberg : M 

SUBJECT: Task Board Notes 

The general approach to designing new tasks has been experimental. 
An attempt to create new tools and techniques along with their 
evaluation has led to generic applications which are closely related to 
scenarios for Space Station environments. The tasks can be used for 
evaluating and studying human performance parameters using the Advanced 
Teleoperator System. The results will provide valuable knowledge needed 
for enhanced capabilities in telerobotics. 

According to a current NASA report on status and future directions 
for telerobotics research and development programs, the following areas 
were identified as possible subjects for developing data to support 
international standards: (1) task boards (2) system performance measures 
on task boards (3) human performance measures on task boards with 
teleoperation and (4) autonomous - telerobo tic- teleopera ted performance 
comparisons. Standardization of task boards will allow different 
techniques and technologies to be evaluated against a common set of 
tasks. This will also permit the selection of consistent and compatible 
approaches to telerobotic applications as well as establish common 
measures of performance. 

The purpose of this memo is to document the unique set of tasks 
which has recently been developed at JPL. A brief outline of the task 
description as well as performance objectives are discussed. Sketches 
of the task components and photographs of the completed tasks are 


provided. 
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PEG- IN-HOLE TASK 


Two versions of the peg- in-hole task have been developed. The 
first version had different sized chamfers and tolerances for each hole, 
as shown in Figure 1. This version was fabricated from 6061 -T6 
aluminum with a 303 stainless steel peg. Figure 2 shows a drawing of 
the aluminum matrix set-up. Preliminary experiments using the peg- in- 
hole task for measuring operator performance have shown that chamfers do 
not improve operator efficiency. Other tests showed no significant 
difference in task completion time for the different hole tolerances. 

A second version was machined without chamfers, but with tighter 
tolerances (a 1.0000 inch peg In a 1.0005 inch hole). The new design 
features hardened sleeves that can be easily changed or modified (see 
Figure 3). The 9-hole matrix block was made from 7075-T7 aluminum 
because it is the "toughest” aluminum available. In other words, the 
material has a very high tensile strength, but not necessarily a large 
hardness value. 7075 -T7 aluminum is also "easy" to machine. The rear 
cover plate (used to keep the sleeves in place) was made from 6061 -T6 
because this aluminum costs less and usually proves more readily 
available. The sleeves were made of A-286 tool steel. Sleeves are 
better for holding tight tolerances because they can be fabricated on a 
lathe and honed to achieve the proper tolerance. Flight fasteners are 
also made of this material. A-286 can be heat treated to obtain a 
higher hardness value, but heat treating was not required for this 
application. In addition, A-286 steel will not rust (an important 
requirement for tight tolerance applications) and has a much higher 
Rockwell hardness value than the machine ground 303 stainless steel peg. 
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TOP: 1.0030 TOP: 1.0019 TOP: 1.0007 

BOTTOM: 1.0032 BOTTOM: 1 .0019 BOTTOM:1.0005 

NO COUNTERSINK NO COUNTERSINK NO COUNTERSINK 



TOP: 1.0028 TOP: 1.0018 TOP: 1.0006 

BOTTOM. 1.0032 BOTTOM: 1 .0016 BOTTOM: 1 .0004 

1 .062 DIR. COUNTERSINK 1 .062 DIR. COUNTERSINK 1 .062 DIR. COUNTERSINK 



TOP: 1.0032 TOP: 1.0016 TOP: 1.0006 

BOTTOM: 1.0032 BOTTOM: 1 .0016 BOTOM: 1.0004 

1 .1 25 DIR. COUNTERSINK 1 .1 25 DIR. COUNTERSINK 1.125 DIR. COUNTERSINK 


DIAMETER 

0.9974 

0.9977 

0.9982 

0.9988 


PEG DIMENSIONS 


.THIS END ENTERED 
HOLE FIRST 


LENGTH =. 4.75 IN. 


DIAMETER 

0.9944 

0.9950 

0.9964 

0.9965 


FIGURE 1: ORIGINAL PEG-IN-HOLE TASK DIMENSIONS 
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SLEEVES WERE MACHINED WITH THE FOLLOWING DIMENSIONS 



1.0002, 1.0003, and 1.0006 in. sleeves were also machined. 
The peg has a diameter of 1.0002 in. and a length of 4.75 in. 

FIGURE 4: NEW PEG-IN-HOLE TASK DIMENSIONS 
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The following material selection possibilities for the peg- in-hole 
task were also considered: 

o Use the original peg- in-hole 9-hole matrix block and add A-286 
tool steel sleeves. 303 or 304 stainless steel for the peg. 
o Use the original peg- in-hole 9-hole matrix block and add 4130 
steel sleeves. Case hardened or tool steel for the peg. 
o Use the original peg- in-hole 9 -hole matrix block and add bronze 
or brass sleeves. 303 or 304 stainless steel for the peg. 
o Fabricate an entirely new block from 4130 steel with no sleeves. 

Case hardened or tool steel for the peg. 
o Fabricate an entirely new block from 303 or 304 stainless steel 
with no sleeves. Bronze or brass for the peg. 

It is important to remember that the block/sleeve and the peg must 
exhibit proper disparity between each other to prevent seize-up. In 
other words, one must be "hard" and the other must be "soft”. For 

example, a 303 stainless steel block with a 303 stainless steel peg is 
not a possibility because gouging, gulling, and cold welding will occur. 
Cold-worked steels are also not acceptable for tight tolerance 
applications because they do not machine to a smooth surface. In 
addition, a 9 -hole matrix block made from 4130 or stainless steel would 
be approximately 2.5 times heavier than one made from aluminum. This 
weight factor would make it very hard and clumsy to change the module 
around the task board. Moreover, 4130 will rust in normal laboratory 
conditions . 
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ELECTRICAL CONNECTORS TASK 


The following four electrical connectors are mounted in the task 
window as shown in Figure 5: 

(1) 3-pole electrical socket (Hewlett Packard 1251-2357). 

(2) 2-pole flanged electrical inlet (General Electric 4343-5). 

(3) 0.25 inch head-phone style socket. 

(4) Cannon 15 pin rectangular connector, 

A corresponding plug attaches to each socket at one end and 
terminates under a clamp within the task module at the other end. 



FIGURE 5: ELECTRICAL CONNECTORS TASK 


100 





BAYONET Ci 




L-IASK 


This task involves the mating and unmating of a bendix PT06A-20- 
16S/16P electrical connector. A white line is painted on the socket 
face in order to "see” the position of the notches for locking the 
socket to the plug. The top camera view of the task board is used for 
checking the connector's orientation, A square grip was fitted to the 
round connector for better handling as shown in Figure 6 below. 



FIGURE 6: BAYONET CONNECTOR TASK 
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An effort is made to minimize unnecessary force while removing and 


attaching the different sized blocks. 



FIGURE 7: VELCRO ATTACHMENT TASK 
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CIRCUIT gOARP BH fQVAL TA S K 


This task contains a 6.5 inch deep x 6.03 Inch width circuit board 
rack (Buckeye STP BPR 6001 -63N) with adjustable height. The rack has 12 
board capacity and 0.348 card spacing. The task features a latching 
door with indexing hinges; see Figure 8. Two stainless steel sliders 
(Figure 9) rotate through two corresponding Delrin guides (Figure 10) . 
The guides employ spring loaded ball plungers (see manufactures parts 
list on the attached page) that detent through holes in the slider. 
The intensity of the indexing can be changed by adjusting the ball 
plungers . 



FIGURE 8: CIRCUIT BOARD REMOVAL TASK 
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CRANK TASKS 


Two types of cranks are used on the task board. The first crank 
has an extension arm and a knob while the other uses a simple one inch 
diameter plastic knob. A nylon bushing with a 10-32 pan head screw are 
used to attach aluminum knob to the aluminum handle. Both cranks 
utilize a rotary switch with adjustable indexing (2 to 12 positions) 
every 30 degrees for "positive action response” to the operator. 

An attempt is made to rotate the cranks as smoothly as possible in 
order to minimize unnecessary force. The crank task, shown in Figure 11 
below, operates throughout an entire 360 degree circle while the knob 
task, shown in Figure 12, functions only within a 300 degree arc. 
Numbers are engraved at each point of index. 
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PLANAR CONTACT 


This task, displayed in Figure 13, includes a special "tool" for 
making "planar contact” with a surface within the task module. The 
tool pushes against four springs mounted in the rear of the task 
structure. When the tool makes complete surface contact with the rear 
task plate, four micro switches are engaged which light up four red 
indicator lamps. If any part of the tool plate Is not making absolute 
contact with the spring/switch plate, the matching light will not turn 
on. Figure 14 shows the tool handle which is an exact inverse of the 
"smart-hand” grippers . 

Two electrical component boxes are mounted on the back side of the 
task structure which house two nine volt batteries and all electrical 
wiring. A switch mounted outside one of the electrical component boxes 
will turn on all lights at once for testing purposes. 


TASK 

WINDOW 

ELECTRICAL 

COMPONENT 

BOX 

4-IN. SQUARE 
TUBING, 4 IN. 
DEEP 


TOOL 

SURFACE 



INDICATOR 
LIGHTS, 4 PLS. 


MICRO 
SWITCHES 
4 PLS. 


TOOL HANDLE 
(SEE FIGURE 14) 


FIGURE 13: PLANAR CONTACT TASK 
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FASTENER REMOVAL/ATTACHMENT TASK 


A distinctive set of tools was procured for this task. Modified 
ratcheting drivers are used to screw various sized bolts into 
corresponding threaded holes. The drivers have custom nylon tool 
handles similar to the one in Figure 14. 

The fasteners are coupled with one task plate (Figure 15) while 
the driver and sockets are stored in a second task plate (Figure 16). 
Sockets can be easily changed without extra assistance. Figure 17 
shows a drawing of the socket containers. The driver also features a 
special T-shaped extension for changing the ratchet direction. This 
can also be performed using only the teleoperator system. 



FIGURE 15: FASTENER REMOVAL/ATTACHMENT TASK 
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FIGURE 16: DRIVER/SOCKET HOLDER SUBTASK 
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Neoprene Rubber Coating for "Smart-Hand" Grippers and Task Board 
Components . 

Numerous components on task board as well as the "smart-hand" 
grippers were coated with a resilient material that provides high 
friction gripping. Plastic Dip, Solathane, Color Guard Tough Rubber 
Coating, and neoprene rubber were all tested. The neoprene rubber 
turned out to be the most durable covering. An outside vendor was used 
to apply the .055 inch thick coating. 
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Appendix C 

Task Sequence Descriptions 
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TASK SEQUENCE DESCRIPTIONS 


1. Velcro Blocks 

1.1 Attach and detach both blocks; tap mark, grasp small block, detach, attach block, 
release grasp, tap on small block, grasp large block, detach, attach block, release 
grasp, tap on large block, release grasp, tap mark. 

1.2 Exchange block positions: tap mark, grasp small block (at position “A”), detach, 
move to side (position “B”), attach, release, tap, grasp large block (position “C”), 
detach, move to position “A”, attach, release, tap, grasp small block (at position 
“B”), move to position “C”, attach, release, tap small block, tap mark. 

2. Peg-in-Hole Matrix 

2. x Insert peg in hole x: (nine holes, subtasks 2.1 - 2.9) start with peg in gripper, tap 

mark with peg tip, move to hole x, insert peg, release grasp, tap on end of peg, 
regrasp, extract peg, move to mark, tap mark. 

3. Electrical Connectors 

3. x Unmat, e and mate connector x. Tap mark, move to connector x, grasp connector, 

muriate, move to mark, tap mark with connector, move back to receptacle, mate, 
release, tap connector, move back to mark, tap. Key: (1) 3-prong chassis power 
connector, (2) 25-pin RS-232 connector (3) l/4 r headphone plug. 

4. Bendix Connector 

4.1 Unmate and mate connector. Tap mark move to left receptacle (where connector 
is mated), align jaws, grasp locking ring (via fixture), rotate counter clockwise 90 
deg, remove connector, move to right receptacle, align connector, rotate locking 
ring clockwise 90 deg, ungrasp, move to mark, tap. 

Subject Instructions. After practice, and before beginning the data taking, the subjects 
were instructed to perform the tasks as fast as possible while minimizing the amount of 
force used. In addition, the subjects were asked to avoid making “errors” of specific types 
defined in Table 3. 
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Appendix D 

Operator Skill Testing and Prediction 
Haya Zak 
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OPERATOR SKILL TESTING AND PREDICTION 


In an attempt to establish a teleoperator skill test that would constitute a prediction 
criterion for operator’s performance, two motor control tests were designed: the “Star 
Test” and the “Circle Test.” 

Test Descriptions 

Both tests were implemented on an IRIS 2400 graphics computer. The Star Test 
had the following features: A figure of a star was presented to the subject, consisting of 
an inner star outline and an outer star outline separated by approximately 0.25 inches 
(Fig. D-l). The objective was to trace between the two star outlines (called “inside the 
star”) with a moused-controlled cursor, while maintaining maximum tracing speed and 
accuracy. Speed was measured in “frames” (in one minute there are 1440 frames), an 
indicator of the total time required to complete an end-to-end tracing of a star. Accuracy 
was measured as “total length” — the length (inches) of the cursor’s path in tracing inside 
and outside the star. The optimal length in tracing the star was 34.38 inches, thus any 
“total length” recording that exceed this value is an indication of whether and by how 
much the subject deviated from the optimal path. The recording of completion time and 
performance accuracy started as soon as the subject pushed the mouse button. As long 
as the cursor moved inside the star, the path traveled by the cursor was filled with a red 
color, serving as a positive feedback to the subject. Once deviation from the inner path 
occurred, the red color would stop filling the inner path and the green line marking the 
path traveled by the cursor went outside the star, serving as a negative feedback. In order 
to “get on the right track again,” the subject had to get on the inner path exactly on or 
before the point of deviation, marked by the end of the red filled part. 

The Circle Test consisted of a target red circle (approximately 1 inch in diameter) 
which was following a circular (invisible) path (approximately 10 inches in diameter) with 
a speed of 22.35 cycles per minute. Another “small” blue circle (approximately 0.5 inch 
in diameter) controlled by the mouse, was the “hunter.” The objective was to keep the 
small circle on target (the big circle) as the target moved along its path. Accuracy was 
measured in this case as time on Target (percent of total time). 

Experimental Procedures 

The 5 subjects performed 32 repetitions of the Star Test and 60 repetitions of the 
Circle Test. The high number of repetitions was designed to bring the subjects to a 
reasonably reliable level of performance, i.e., where the “leveling off’ of the learning curve 
occurs. The Star Test was further elaborated to include various mouse orientations — 
either 0, 90, 180, or 270 degrees in relation to the hand’s grasp position. Subjects were 
required to change the mouse orientation every repetition, such that 8 repetitions were 
performed for each orientation. 
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Figure D-l. Screen image from the star tracing test used in motor control pre-testing. The 
subjects used a mouse to trace between the two star outlines. The task was made more difficult 
by imposing rotations of 90, 180, 270 and 0 degrees between mouse motion and cursor motion. 
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Data Analysis 

The two mean performance measures for the Star Test (total length and completion 
time) were taken for each subject, then correlated with teleoperation performance measures 
(time, force, and errors). A summary of results is shown in Tables D.1-D.8. The Pearson 
product-moment correlation was computed for each test- teleoperation performance mea- 
surement pair. In this method, the significance of each correlation is determined via the t 
test (where t = ^”~, ). For 4 subjects (at alpha=0.05), the value required for significance 
is f=0.95. The numbers in each square indicate the number of significant correlations out 
of the total number of correlations (the latter in parenthesis). 

Table D.l summarizes our findings for the Star Test. The data breakdown is by mouse 
orientation, by mode (FFB or MANUAL) and by task (1-4 and ALL). Since other kinds 
of analysis performed on this data show task 4 to be quite different in nature from tasks 
13, it was decided to treat task 4 as a special category, while tasks 1-3 could be averaged 
together to form the ALL category. From table D.l it appears that there were no significant 
correlations, i.e., 0 out of 80, for the MANUAL mode, while 6 correlations out of 8S were 
found to be significant in the FFB mode. Nevertheless, 6(88) may still be random results 
(for alpha=0.1), therefore, it is difficult to conclude that the FFB mode contributed to 
the results. Within the FFB mode, the most “promising” orientation appears to be 180 
degrees, but again 3(22) significant correlations may be due to chance alone. No particular’ 
task appears to be taking the lead in terms of significance. The ALL category seems to 
correlate slightly more than the individual tasks, yet this may very well be a chance result 
as well. Overall, only 6(168) scored significantly, thus, the results do not allow us to reject 
the null hypothesis at alpha— 0.05. 

Tables D.4-D.S summarize the Star Test results for each task (Table D.5, task 1; Table 
D.6, task 2; Table D.7, task 3; Table D.8, task 4; Table D.4 average of tasks 1-3). The 
new information gained by this representation is that, “time” for the Star Test may be a 
better predictor of teleoperation measures: 7(84) were significant for “time” while 0(84) 
was recorded for “length.” The breakdown of ERRORS by task is not available, although 
it does exist for all tasks (1-4). Since 2(8) correlations with ERRORS were found to be 
significant, this is a promising result that should be explored in future experiments. 

Table D.2 summarizes the results for the Circle Test. While for the FFB mode there 
are no significant correlations found, i.e., 0(5) for the TIME and ERRORS, 2(4) correla- 
tions were significant for the FORCE measure. The latter result offers good support for 
rejecting the null hypothesis. Once again, comparison of the FFB mode to the MANUAL 
mode results in FFB outperforming the MANUAL in terms of the number of significant 
correlations. 

Table D.3 is yet another summary of results for the Circle Test. 
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Concluding Remarks 

The goal of establishing a valid teleoperator skill test has not yet been achieved based 
on the results of the above experiment. Further research is needed to investigate the 
potential of the Star Test and the Circle Test for being reliable teleopcrator skill tests. 
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TABLE D.l 



TASK 

STAR 0° 

STAR 90° 

STAR 180° 

STAR 270° 

total 

FFB 

MODE 

1 

0(4) 

0(4) 

0(4) 

0(4) 

0(161 

2 

0 (4) 

0 (4) 

1 (4) 

0 (4) 

■IAMB 

3 

1(4) 

0 (4) 

0(4) 

0(4) 

1 (16) 

4 

1 (4) 

0 (4) 

0(4) 

0(4) . . 

1 (16) 

ALL* 

0 (6) 

0(6) 

2(6) 

1 (6) 

Han 

subtotal 

2 (22) 

0 (22) 

3 (22) 

1 (22) 


MANUAL 

MODE 

1 

0(4) 

0 (41. 

0 (4) 

0 (4) 

0 (16) 

2 

0 (4) 

0 (4) 

0(4) 

0 (4) 

0 (16) 

3 

0 (4) 

0 (4) 

0 (4) 

0 (4) 

0 (16) 

4 

0 (4) 

0 (4) 

0 (4) 

0(4) 

0(16) 

ALL* 

.. 0(4) 

0(4) 

0 (4) 

0 (4) _ 

0 (16) 

subtotal 

0 (20) 


0 (20) 

0 (20) 

0 (80) 


total 

2 (42) 

0 (42) 

3(42) 

1 (42) 

6(168) 


TABLE D . 2 





1 

1 

1 

msmm 

total 

FFB 

MODE 

TIME 

0 (1) 

0 (1) 

0 (1) 

0 (1) 

0 (1) 

0 (4) 

FORCE 

0 (1) 

i (i) 

0(1) 

0 (1) 

1 (1) 

2 (4) 

ERRORS 

n/a 

n/a 

n/a 

n/a 

0(1) 

0 (1) 


0(2) 

1(2) 

0 (2) 

0 (2) 

113 ) 

2 (.11) 

MANUAL 

MODE 

TIME 

0(1) 

0 (1) 

0(1) 

0(1) 

ran 

0(5) 

FORCE 

0 (1) 

0 (1) 

0 (1) 

0 (1) 

0 (1) 

0 (5) 

subtotal 

0(2) 

0 (2) 

0(2) 

0(2) 

0(2) 

0 (10) . 


total 

0 (4) 

1 (4) 

0 (4) 

0(4) 

1 (5) 

2 (21) 


TABLE D . 3 



TIME 

FORCE 

ERRORS 

total 

FFB 

MODE 

0 (5) 

2 (5) 

0(1) 

2 (11) 

MANUAL 

MODE 

0(5) 

0 (5) 

n/a 

0 (10) 

total 

0 (10) 

2 (10) 

0 (1) 

2 (21) 


ALL includes tasks 1-3 only, except for the correlations with 
ERRORS where all tasks (1-4) are included. 


119 















































































































































TABLE D . 4 


TIME 


EEB OH 

LENGTH MANUAL 0 ( 4 
subtotal 0(8) 


0 (4 


MANUAL 0(4) 


0 (8 


total 0 (16) 



FORCE 


0 <4 


0 


0 ( 8 ) 


1 


0 (4) 


1 


1 (16) 


ERRORS I total 


>1 

0 (4 

u 

0 (12 


n/a 


0 (4) 


2 


n/a 


2 


2 


0 


0 


3 (12 


0 ( 8 ) 


3 (20 


3(40) 



TIME 


0 (4 


0 (4 


subtotal 0(8) 


0 (4 


MANUAL I 0(4) 


0 (8 


total I 0(16) 


FORCE 


0 (4 


0 (4 


0 ( 8 ) 


2 


0 


2 8 


2 (16) 


ERROR 


n/a 


n/a 


n/a 


n/a 


n/a 


0 (8 


o (8: 


0(16) 


2 


0 ( 8 ) 


ll 

n/ 

a 

1 2(16; 


2 (32) 





FORCE 


0 (4 


0 


MANUAL I 0(4) 


1 (8 


total I 1 (16) 


0 (8) | 

n/a 

0 (4 

rn 

n/a 

0 (4) 

n/a 

0 (8 

rn 

n/a 

0(16) 

n/a 


0 (8 


0 (8 


0 (16) 


1 


0 ( 8 ) 


1 (16 


1 (32) 



TIME 


FFR I 0(4 
LENGTH MANUAL 0(4 
s ub t ota ll 0 (8) 






total I 1(16) 


0 (4 


0 (4 


0 ( 8 ) 


0 (4 



0(8 


0 (16) 


ERROR 


a 


n/a 


n/a 


a 


n/a 


a 


n/a 


■ 


0 (8 


0 


0 (16) 


1 (8 


0 ( 8 ) 


1 (16 


1 (32) 
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Appendix E 

Typical Force/Torque Records for Additional Tasks 
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X Axis Force Y Axis Force Z Axis Force (lb) 


0 

-10 

10 

0 

-10 

10 

0 


-10 

Time (seconds) 


Figure E-l. Raw data acquired from a single repetition of the velcro experiment (continued 
in figures E-2 and E-3). X, Y, and Z axis forces measured in the robot hand reference frame. 
Spikes represent taps made intentionally by the operator to provide reference benchmarks in the 
force signal. 



Roll Torque Yaw Torque Pitch Torque (in-lb) 


40 
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Figure E-2. Pitch, yaw and roll torques during the velcro task. 
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Gripping Force (lb) Jaw Opening 


256 


224 - 

192 1 

160 - 
128 - 
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Figure E-3. Jaw opening and finger gripping force for the velcro task. 
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X Axis Force Y Axis Force Z Axis Force (lb) 


10 





Figure E-4. Raw data acquired from a single repetition of the 25 pin connector experiment, con- 
tinued in Figures E-5 and E-6. 
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Roll Torque Yaw Torque Pitch Torque (in-lb) 



Time (seconds) 


Figure E-5. Pitch, yaw and roll torques during the 25 pin connector task. 
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Gripping Force (lb) Jaw Opening 
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Figure E-6. Jaw opening and finger gripping force for the 25 pin connector task. 
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X Axis Force Y Axis Force Z Axis Force (lb) 


10 



Time (seconds) 


Figure E-7. Raw data acquired from a single repetition of the bayonet connector experiment, 
continued in figures E-8 and E-9. 
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Gripping Force (lb) Jaw Opening 



Figure E-9. Jaw opening and finger gripping force for the 25 pin connector task. 


130 



1. Report No. 89-18 


TECHNICAL REPORT STANDARD TITLE PAGE 


2. Government Accession No. I 3. Recipient'* Catalog No. 


4. Title and Subtitle 


Performance Evaluation of a 

Six-Axis Generalized Force- 
KH1X Reflecting Teleoperator 


7. Author (s) 


B. Hannaford et al. 


9. Performing Organization Name and Addre» 

JET PROPULSION LABORATORY 
California Institute of Technology 
4800 Oak Grove Drive 
Pasadena, California 91109 


12. Sponsoring Agency Name and Address 

NATIONAL AERONAUTICS AND SPACE ADMINISTRATION 
Washington, D.C. 20546 

15. Supplementory Notes 


5. Report Date 

June 15, 1989 


6. Performing Organization Code 


8. Performing Organization Report No. 


10. Work Unit No. 


11. Contract or Grant No. 

NAS7-918 


13. Type of Report and Period Covered 


14. Sponsoring Agency Code 


16 Abstract Recent Work at JPL in real-time distributed computation and contro ) haS 
‘culminated in a prototype force-reflecting telemanipulation system hjjvLng d ^“ ila 
master (cable-driven force-reflecting hand controller) and slave (PUMA 560 robot 
with custom i controller), extremely high sampling rate (1000 Hz), and low 1°°P 
computation delay (5 msec). In a series of experiments with this system and five 
trained test operators covering over 100 hours of teleoperation, h 

performance in a series of generic and application-driven tasks with and witho 
force feedback, and with control shared between teleoperation ^d local sensor 
referenced control. Measurements defining task performance include 100 Hz 
recording of six-axis force/torque information from the slave manipulator wrist, 
task completion time, and visual observation of predefined task errors. 

The tasks consisted of high precision peg-in-hole insertion, elect ®°^® ctors » 
velcro attach-de-attach, and a twist-lock multi-pin connector. Each task w 
repeated three times under several operating conditions: no ™>al bilateral 

telemanipulation, forward position control without force feedback and shared 
control." in shaied control, orientation was locally servo control ed to comply with 
applied torques, while translation was under operator control. All performance 
measures improved as capability was added along a spectrum of capabilities 
ranging from pure position control through force-reflecting teleoperation and 
shared control. Performance was optimal for the bare-handed operator. 


17. Key Word* (Selected by Author(*)) 

Teleoperator, teleoperator performance, 

telemanipulation, force-reflecting tele- 
manipulation system, six-axis generalized 
force-reflecting telemanipulation 


18. Distribution Statement 

Unclassified — Unlimited 


19. Security Cla**if. (of thi* report) 20. Security Classif. (of this poge) 

Unclassified Unclassified 


21 . No. of Pages 

130 


22. Price 


jpl ois4 r eita 




/ 

w 

/ 

/ 



